Methods and systems for imaging skin using polarized lighting

ABSTRACT

An imaging system for imaging skin includes a light source to illuminate a subject and a first polarizer to polarize light provided by the light source to illuminate the subject. The imaging system also includes a photodetector to acquire an image of the subject as illuminated by the light source and an adjustable second polarizer, coupled to the photodetector, to provide an adjustable degree of polarization of light received by the photodetector.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 61/164,356, titled “Methods and Systems for Analyzing SkinConditions Using Polarized Light,” filed Mar. 27, 2009, which is herebyincorporated by reference in its entirety.

TECHNICAL FIELD

The disclosed embodiments relate generally to imaging and analyzing skinconditions, and more particularly, to imaging and analyzing skinconditions using polarized lighting.

BACKGROUND

High-quality images of a subject's skin have potential applications indermatology and cosmetics, among other fields. Obtaining high-qualityskin images, however, presents significant engineering challenges. Forexample, skin conditions on the surface of the skin, such as wrinkles,can interfere with imaging sub-surface features. In another example,skin care products can interfere with images taken using fluorescencetechniques.

SUMMARY

In some embodiments, an imaging system for imaging skin includes a lightsource to illuminate a subject and a first polarizer to polarize lightprovided by the light source to illuminate the subject. The imagingsystem also includes a photodetector to acquire an image of the subjectas illuminated by the light source and an adjustable second polarizer,coupled to the photodetector, to provide an adjustable degree ofpolarization of light received by the photodetector.

In some embodiments, a method of generating a sub-surface skin imageincludes illuminating a subject with polarized light having a firstpolarization. An adjustable polarizer is set to a first setting to admitlight having the first polarization onto a photodetector and otherwisereject light. With the adjustable polarizer in the first setting, thephotodetector is used to acquire a first image of the illuminatedsubject. The adjustable polarizer is set to a second setting to at leastpartially reject light having the first polarization and to at leastpartially admit light having polarization distinct from the firstpolarization onto the photodetector. With the adjustable polarizer inthe second setting, the photodetector is used to acquire a second imageof the illuminated subject. The first image is subtracted from thesecond image to generate a third image of the subject.

In some embodiments, a computer-implemented method of processing anddisplaying images of skin includes receiving a first image of a subject.The first image was acquired at an optical apparatus with the subjectilluminated with light having a first polarization and the opticalapparatus configured to receive light having the first polarization andto otherwise reject light. A second image of the subject is received.The second image was acquired at the optical apparatus with the subjectilluminated with light having the first polarization and the opticalapparatus configured to at least partially reject light having the firstpolarization and to at least partially receive light having polarizationdistinct from the first polarization. The first image is subtracted fromthe second image to generate a third image of the subject. The thirdimage is displayed.

In some embodiments, a system for processing and displaying images ofskin includes memory, a display, one or more processors, and one or moreprograms stored in the memory and configured for execution by the one ormore processors. The one or more programs include instructions toreceive a first image of a subject. The first image was acquired at anoptical apparatus with the subject illuminated with light having a firstpolarization and the optical apparatus configured to receive lighthaving the first polarization and to otherwise reject light. The one ormore programs also include instructions to receive a second image of thesubject. The second image was acquired at the optical apparatus with thesubject illuminated with light having the first polarization and theoptical apparatus configured to at least partially reject light havingthe first polarization and to at least partially receive light havingpolarization distinct from the first polarization. The one or moreprograms further include instructions to subtract the first image fromthe second image to generate a third image of the subject andinstructions to display the third image.

In some embodiments, a computer readable storage medium stores one ormore programs configured to be executed by a computer system to processand display images of skin. The one or more programs includeinstructions to receive a first image of a subject. The first image wasacquired at an optical apparatus with the subject illuminated with lighthaving a first polarization and the optical apparatus configured toreceive light having the first polarization and to otherwise rejectlight. The one or more programs also include instructions to receive asecond image of the subject. The second image was acquired at theoptical apparatus with the subject illuminated with light having thefirst polarization and the optical apparatus configured to at leastpartially reject light having the first polarization and to at leastpartially receive light having polarization distinct from the firstpolarization. The one or more programs further include instructions tosubtract the first image from the second image to generate a third imageof the subject and instructions to display the third image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic cross-sectional view of polarized light incidenton and reflected from skin in accordance with some embodiments.

FIGS. 2A-2C are block diagrams of imaging systems for imaging skin inaccordance with some embodiments.

FIG. 3 is a diagram illustrating an imaging system including a light boxin accordance with some embodiments.

FIG. 4 is a block diagram illustrating a system in which a networkcouples an acquisition system to an image processing system and acosmetic formulation system in accordance with some embodiments.

FIG. 5 is a block diagram illustrating an automated formulation systemthat includes a cosmetic formulation control system coupled to anautomated cosmetic formulator in accordance with some embodiments.

FIGS. 6A and 6B are diagrams illustrating data structures for analyzingimages of skin in accordance with some embodiments.

FIGS. 7A-7C are flow diagrams illustrating a method of generating asub-surface skin image in accordance with some embodiments.

FIG. 7D is a flow diagram illustrating a computer-implemented method ofprocessing and displaying images of skin in accordance with someembodiments.

FIG. 8 is a block diagram illustrating a computer in accordance withsome embodiments.

Like reference numerals refer to corresponding parts throughout thedrawings.

DESCRIPTION OF EMBODIMENTS

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea thorough understanding of the present invention. However, it will beapparent to one of ordinary skill in the art that the present inventionmay be practiced without these specific details. In other instances,well-known methods, procedures, components, and circuits have not beendescribed in detail so as not to unnecessarily obscure aspects of theembodiments.

FIG. 1 is a schematic cross-sectional view of polarized light incidenton and reflected from skin in accordance with some embodiments. A lightsource 102 is covered by a polarizer 104, such that light 108illuminating skin 100 has a particular polarization 110. In the exampleof FIG. 1, the polarizer 104 is a linear polarizer and the light 108 islinearly polarized (i.e., polarization 110 is linear polarization).Light 112 is reflected from the surface of the skin 100 and detected bya camera (e.g., a digital camera) 106. The light 112 reflected from thesurface of the skin 100 has the same polarization 110 as the incidentlight 108 and thus is also linearly polarized. Not all of the incidentlight 108 is reflected from the surface of the skin 100, however. Aportion 114 of the incident light 108 penetrates to a particular depthwithin the skin 100 before being reflected. (For simplicity, FIG. 1shows incident light 108 penetrating to a single depth within the skinbefore being reflected as light 116 and 118. In reality, incident light108 penetrates to a range of depths before being reflected.) The light118 reflected from beneath the surface of the skin 100 has apolarization (e.g., an elliptical polarization) 120 distinct from thepolarization 110 of the light 112 reflected from the surface of the skin100. In general, the polarization 120 of the light 118 is random.

The camera 106 thus receives partially polarized light: a portion of thereceived light has the polarization 110, and thus corresponds to light112 reflected from the surface of the skin 100, while another portionhas essentially random polarization 120, and thus corresponds to light118 reflected from beneath the surface of the skin 100.

The camera 106 may be equipped with a polarizer which may be configured(e.g., by rotating the polarizer) to (1) admit only light having thepolarization 110, such that all other polarizations are rejected, (2)reject all light having the polarization 110, such that admitted lightis polarized perpendicular to the polarization 110, or (3) admitpartially polarized light that includes components having thepolarization 110 and components having a polarization perpendicular tothe polarization 110. In the first case, an image taken by the camera106 corresponds to light reflected from the surface of the skin 100 andis thus an image of the surface of the skin 100. In the second case, animage taken by the camera 106 corresponds to light reflected from adepth beneath the surface of the skin 100 that varies from approximately350 microns for very dark skin to approximately 3 mm for very fair skin.The image in the second case is thus a sub-surface image of the skin100. In the third case, an image taken by the camera 106 corresponds tolight reflected from both the surface and from varying depths beneaththe surface of the skin 100 and thus can be considered a combination ofsurface and sub-surface skin images.

Referring to the component of received light with the polarization 110as PAR (i.e., polarized parallel to a plane of polarization of theincident light 108) and to the component of received light polarizedperpendicular to the plane of polarization of the incident light 110 asPER, the degree of partial polarization of light admitted by thepolarizer and thus imaged by the camera 106 can be quantified as:Degree of Partial Polarization=(PAR−PER)/(PAR+PER)  (1)This formula thus quantifies the percentage of light admitted by thepolarizer that corresponds to light reflected from the surface of theskin 100 as opposed to light reflected from beneath the surface of theskin 100.

FIG. 2A is a block diagram of an imaging system 200 for imaging skin 203of a subject 202 in accordance with some embodiments. The imaging system200 images the skin 203 in accordance with the physical principlesillustrated in FIG. 1. While the system 200 is illustrated as imaginghuman facial skin, in some embodiments the system 200 may be used toimage any type of animal skin. In the system 200, a camera (e.g., adigital camera) 204 includes a photodetector 216 to acquire images ofthe subject 202, computer memory 212 to store acquired images, andcamera control circuitry 214 (e.g., one or more processors) to controlacquisition and storage of the images. The photodetector 216, memory212, and control circuitry 214 are contained in a housing 206 of thecamera. In some embodiments, the photodetector 216 comprises an array ofcharge-coupled devices (CCD), charge-injection devices (i), or CMOSdevices. In some embodiments, the photodetector 216 includes 5-15 ormore megapixels. In some embodiments, each pixel includes threesub-pixels corresponding to three distinct color channels (e.g., red,green, and blue, or alternatively, a set of colors associated withanother color space). In some embodiments, the photodetector 216 isrotatable to provide a variable aspect ratio for acquired images.Rotation of the photodetector 216 is controlled, for example, by thecontrol circuitry 214.

The system 200 includes one or more light sources 208 (hereinafter,“light sources 208”) to illuminate the subject 202 and one or morepolarizers 210 (hereinafter, “polarizers 210”) to polarize the lightfrom the light sources 208 illuminating the subject 202. In someembodiments, the light sources 208 and polarizers 210 are coupled to thecamera housing. For example, the light sources 208 and polarizers 210may be affixed to the camera housing 206, as illustrated in FIG. 2A, orintegrated into the camera housing 206. Alternatively, the light sources208 and polarizers 210 may be physically separate from the camera 204.In some embodiments, the light sources 208 include one or more flashbulbs, one or more light-emitting diodes (LEDs), or one or morefluorescent high-temperature white-light sources. In some embodiments,the polarizers 210 include one or more linear polarizers. If multiplepolarizers 210 are present, the multiple polarizers 210 are aligned toprovide the same polarization. In some embodiments, the polarizers 210are fixed, such that the polarization they provide is not adjustable. Apolarizer 210 may be mounted on a respective light source 208 orotherwise arranged such that it polarizes light from the light source208 that is incident on the subject 202.

The camera 204 includes a lens 218 to focus light onto the photodetector216. In some embodiments the lens 218 is a zoom lens that providesvariable heightened image resolution. The zoom lens may be motorized andcontrolled by associated control circuitry (e.g., included in thecontrol circuitry 214) or may be manually adjustable. The highresolution provided by a zoom lens enables accurate measurement ofimaged skin features (e.g., pore size, hair strands, hair follicles,spots, and moles).

An adjustable polarizer 220 is rotatably mounted on the lens 218 andthereby coupled to the photodetector. In some embodiments, the polarizer220 is an elliptical polarizer, or a circular polarizer, or a linearpolarizer. Rotating the polarizer 220 provides an adjustable degree ofpolarization of light received by the photodetector 216. In someembodiments, a motor 222 attached to the polarizer 220 rotates thepolarizer 220 (e.g., in defined angular increments) in response toinstructions from polarizer control circuitry on a control board 224coupled to the motor 222 via one or more signal lines 238.Alternatively, a knob 242 allows manual adjustment of a degree ofrotation of the polarizer 220, as illustrated in the imaging system 240(FIG. 2B) in accordance with some embodiments.

The polarizer 220 may be adjusted such that it is aligned with thepolarizers 210 and thus only admits light with the same polarization aslight from the light sources 208 as filtered by the polarizers 210. Inthis configuration, the polarizer 220 is said to have 0° rotation withrespect to the polarizers 210. With the polarizer 220 in thisconfiguration, the photodetector 216 may acquire an image of the subject202 corresponding to light reflected from the surface of the subject'sskin 203.

The polarizer 220 may be adjusted such that it is rotated 90° withrespect to the polarizers 210. In this configuration, the polarizer 220rejects all light with the polarization provided by the polarizers 210and admits light having a perpendicular polarization. With the polarizer220 in this configuration, the photodetector 216 may acquire asub-surface skin image of the subject 202 (e.g., corresponding to lightreflected from a depth beneath the surface of the subject's skin 203that varies from approximately 350 microns for very dark skin toapproximately 3 mm for very fair skin).

The polarizer 220 may be adjusted such that it is rotated between 0° and90° with respect to the polarizers 210. In this configuration, thepolarizer 220 admits partially polarized light in accordance withEquation (1). With the polarizer 220 in this configuration, thephotodetector 216 may acquire an image of the subject 202 correspondingto a combination of surface and sub-surface skin images. This image maybe processed to produce a sub-surface skin image by subtracting an imagetaken with 0° rotation of the polarizer 220.

In some embodiments, an imaging system includes a light shield 252 toshield the subject 202 from ambient light, as illustrated for theimaging system 250 (FIG. 2C) in accordance with some embodiments. In thesystem 250, the camera 204 is mounted on a back wall of the light shield252, which extends outward from the camera housing 208 with afrusto-conical shape. By shielding the subject 202 from ambient light,the light shield ensures that most of the light reflected from thesubject 202 and received at the photodetector 216 originated from thelight sources 208 and was filtered by the polarizers 210.

A computer 226 (FIGS. 2A-2B) is coupled to the camera 204 and controlboard 224 via one or more signal lines 236. The computer 226 includesmemory 228 and one or more processors 230 as well as a monitor 232 fordisplaying a user interface (UI) 234. The UI 234 displays acquiredand/or processed images as well as data calculated from acquired and/orprocessed images. In some embodiments, the computer 226 providesinstructions to the control board 224 to rotate the polarizer 220,instructions to the camera 204 to adjust the zoom lens 218, andinstructions to the camera 204 to acquire an image (i.e., to take apicture). The computer 800 (FIG. 8, below) illustrates an example of animplementation of the computer 226 in accordance with some embodiments.

In some embodiments, the functionality of the computer 226 is integratedinto the camera 204. In some embodiments, the camera 204 includes adisplay for viewing acquired and/or processed images as well as datacalculated from acquired and/or processed images.

In some embodiments, the light sources 208, polarizers 210, and camera204 (including polarizer 220) are mounted in an imaging box 302, asillustrated for the imaging system 300 (FIG. 3). The imaging box 302,shown as mounted on a cart 310 for mobility, serves as a light shield toshield the subject from ambient light. First and second light sources208-1, 208-2 and first and second polarizers 210-1, 210-2 are mounted ona rear wall of the box 302, opposite from a chin rest 306 and foreheadpad 304 for receiving the subject's head. An example of such an imagingbox 302 is the Facial Stage DM-3 commercially available from MoritexCorporation of Tokyo, Japan. The system 300 also includes a printer 308for printing acquired and/or processed images as well as data calculatedfrom acquired and/or processed images.

In some embodiments, a reference material is included in acquired imagesto measure light source intensity output change and color change overtime (e.g., resulting from drift in a light source 208). For example, astandard color chart such as the GretagMacbeth ColorChecker may beplaced in the field of imaging (e.g., beneath the chin of the subject202) and used to calibrate the photodetector 216 and/or to post-processacquired images to adjust pixel values based on comparison to knownpixel values for colors in the color chart. Furthermore, imageprocessing software may be used to correct for optical aberrations.

Skin pixels in surface or sub-surface skin images (e.g., imagesgenerated using an imaging system 200, 240, 250, or 300, FIGS. 2A-2C and3) may be analyzed to identify at least one skin condition by comparingpixel values to predetermined criteria associated with various skinconditions. Conditions associated with the skin that may be detected andclassified include, but are not limited to, skin tone/color, pigmentevenness, pigment darkness, diffuse redness (e.g., indicative ofsensitive or reactive skin), intense localized red levels (e.g.,indicative of vascular lesions/telangiectasias), radiance intensity,enlarged pores, roughness variation, emerging lines, fine lines,wrinkles, UV damage, pore health, hydration levels, collagen content,skin type, topical inflammation or recent ablation, keratosis, deeperinflammation, sun spots, different kinds of pigmentation includingfreckles, moles, growths, undereye circles, scars, acne, fungi, erythemaand other artifacts. In addition, image pixels may be used to performfeature measurements, such as the size or volume of a lip, nose, eyes,ears, chin, cheeks, forehead, eyebrows, teeth, or other features. Otherexamples of feature measurements, including pore size measurements, spotcounts, and measurement of the length, thickness and/or curvature of aneyelash, can be made based on information from image pixels. Imagepixels may be used to characterize lip conditions, which may include,without limitation, lip surface area, color, fine lines, wrinkles, andcharacteristics associated with lip edge demarcation. Characteristicsassociated with lip edge demarcation may include, for example, colorcontrast, line roughness, and color variation.

In some embodiments, a skin condition look-up table 600 (FIG. 6A) isused to identify skin conditions. A color value and an intensity valueis measured for a respective skin pixel or group of skin pixels andcompared against color and intensity values for various skin conditionsas stored in the table 600. The table 600 includes a row 602 for eachrespective skin condition stored in the table 600. Each row includesfields that specify a name 602 of a respective skin condition as well asthe minimum color value 606, maximum color 608, minimum intensity value610, and maximum intensity value 612 associated with the respective skincondition. If the measured color and intensity values match the valuesspecified in a row 602, the respective skin condition corresponding tothe row is identified.

In some embodiments, to analyze either skin pixels or non-skin pixels(e.g., pixels corresponding to hair, clothing, eyes, lips, etc.) insurface or sub-surface skin images, pixels are analyzed on apixel-by-pixel basis to distinguish skin pixels from non-skin pixels.Identification of skin and non-skin pixels is described, for example, inU.S. Pat. No. 7,454,046, entitled “Method and System for Analyzing SkinConditions Using Digital Images,” issued Nov. 18, 2008, which is herebyincorporated by reference herein in its entirety. For example, assumingthe pixels have red, green, and blue sub-pixels with pixel values thatrange between 0-255, pixels with red channel values in the range of105-255, green channel values in the range of 52-191, and blue channelvalues in the range of 32-180 are identified as skin pixels.Furthermore, in some embodiments a pre-stored template or coordinatereference is used to define certain pixels as non-skin pixels and a skinmap or skin mask may be used to define certain pixels as non-skinpixels, as described in U.S. Pat. No. 7,454,046 in accordance with someembodiments.

In some embodiments, a surface skin image is compared to a sub-surfaceskin image to compare surface and sub-surface skin conditions. Forexample, surface and sub-surface pigmentation may be compared.

In some embodiments, a sub-surface image may be used alone to analyzepigmentation or other skin conditions. Sub-surfaces images excludewrinkles on the surface of the skin, which can interfere with imaging ofpigmentation. Sub-surface images also exclude glare from the surface ofthe skin, which also can interfere with imaging of pigmentation andother skin features or conditions. Accordingly, sub-surface images canprovide a more accurate indication of skin tone or color than surfaceimages, and can provide a more accurate indication of other skinconditions as well.

In some embodiments, images (either surface or sub-surface) generated byan imaging system (e.g., an imaging system 200, 240, 250, or 300, FIGS.2A-2C and 3) are compared with old (i.e., historical) images stored inmemory to identify variations in skin conditions and features over time.For example, a newly generated image may be displayed next to a storedhistorical image in a user interface (e.g., UI 234, FIGS. 2A-2B). Acomputer system (e.g., the computer 226, FIGS. 2A-2B) performs automatedcomparison of one or more newly generated images with one or morehistorical images to track changes in skin conditions and features. Forexample, the system calculates changes in pigmentation (e.g., skin tone)and changes in size or color of features on the skin. Results of thisautomated comparison are displayed in a user interface (e.g., UI 234,FIGS. 2A-2B).

When comparing multiple images, the images are first aligned to allowthe same features to be identified in the multiple images. In someembodiments, images are aligned using a three-point selection processthat identifies points in the center of the eyes and the center of thelips and aligns the images accordingly.

In the imaging systems 200, 240, and 300 (FIGS. 2A-2B and 3) thecomputer 226 is directly connected to the camera 204. In someembodiments, however, an acquisition system 402 that includes a camera204 is coupled to an image processing system 406 through a network 404.FIG. 4 is a block diagram illustrating a system 400 in which a network404 couples an acquisition system 402 to an image processing system 406in accordance with some embodiments. The network 404 may be any suitablewired and/or wireless network and may include a local area network(LAN), wide area network (WAN), virtual private network (VPN), theInternet, metropolitan area network (MAN), or any combination of suchnetworks. The image processing system 406 may perform various types ofprocessing related to images acquired by the acquisition system,including without limitation subtracting a first image from a secondimage to generate a sub-surface image, analyzing pixel data to identifyskin or feature conditions, and comparing images.

In some embodiments, a system such as the computer 226 (FIG. 2A-2B) orimage processing system 406 (FIG. 4) generates a recommendation for acosmetic product based on analysis of skin pixels. For example, thesystem may analyze skin color in a sub-surface image and recommend acosmetic product based on the skin color. In some embodiments, therecommendation is displayed to the subject 202 (e.g., in the UI 234,FIGS. 2A-2B) or printed out (e.g., using the printer 308, FIG. 3). Insome embodiments, the recommendation is provided at the point of sale(POS) where the subject may buy the recommended product or is displayedon a web page that the subject may use to order the recommended product.

In some embodiments, the image processing system 406 (or alternativelythe acquisition system 402) transmits the recommendation (e.g., throughthe network 404) to a cosmetic formulation control system 408 coupled toan automated cosmetic formulator 410. The formulator 410 then preparesthe recommended product in real time, thus providing the subject with acustomized cosmetic product based on the recommendation.

In some embodiments, the image processing system 406 and cosmeticformulation control system 408 are integrated into a single system.

FIG. 5 is a block diagram illustrating an automated formulation system500 that includes a cosmetic formulation control system 408 coupled toan automated cosmetic formulator 410 in accordance with someembodiments. The control system 408, which includes memory 506 and oneor more processors 504, receives a recommendation for a customizedcosmetic product through a network connection 502. Based on therecommendation, the control system 408 determines a formula for thecustomized cosmetic product (e.g., using a look-up table stored in thememory 506) and provides instructions to the automated cosmeticformulator 410 via one or more signal lines 508 to mix the customizedcosmetic product. Alternatively, the formula is provided to the system500 in the recommendation. An automated dispenser 516 in the formulator410 dispenses one or more base compounds 510, dispersions 512, andadjuvants 514 to an automated mixer 518 in accordance with the formula.The mixer 518 mixes the base compounds 510, dispersions 512, andadjuvants 514 and provides the mixture to an automated packaging unit520, which packages the mixture and dispenses it. In some embodiments,the system 500 is located at the POS. Alternatively, the customizedcosmetic product provided by the system 500 may be shipped to thecustomer.

FIG. 6B is a diagram illustrating a data structure of a cosmetic productrecommendation table 630 used to generate a recommendation for acosmetic product based on analysis of skin pixels. A color value and anintensity value is measured for a respective skin pixel or group of skinpixels and compared against color and intensity values for variouscosmetic products as stored in the table 630. The table 630 includes arow 632 for each respective cosmetic product stored in the table 630.Each row 632 includes fields that specify a respective cosmetic product634 as well as the minimum color value 636, maximum color 638, minimumintensity value 640, and maximum intensity value 642 associated with therespective skin condition. If the measured color and intensity valuesmatch the values specified in a row 632, the respective cosmetic productcorresponding to the row is recommended.

FIG. 7A is a flow diagram illustrating a method 700 of generating asub-surface skin image in accordance with some embodiments. In themethod 700, a subject is illuminated (702) with polarized light having afirst polarization. For example, the subject 202 (FIGS. 2A-2C) isilluminated using light from one or more light sources 208 as filteredby one or more polarizers 210.

An adjustable polarizer (e.g., polarizer 220, FIGS. 2A-2C) is set (704)to a first setting (e.g., a 0° rotation with respect to the polarizers210) to admit light having the first polarization onto a photodetector(e.g., photodetector 216, FIGS. 2A-2C) and otherwise reject light. Withthe adjustable polarizer in the first setting, the photodetector is used(706) to acquire a first image of the illuminated subject. The firstimage thus corresponds to light reflected from the surface of thesubject's skin.

The adjustable polarizer is set (708) to a second setting to at leastpartially reject light having the first polarization and to at leastpartially admit light having polarization distinct from the firstpolarization onto the photodetector. The second setting thus correspondsto a degree of rotation greater that 0° with respect to the polarizers210. With the adjustable polarizer in the second setting, thephotodetector is used (710) to acquire a second image of the illuminatedsubject. The second image thus at least partially includes sub-surfaceimage data.

The first image is subtracted (712) from the second image to generate athird image of the subject. This subtraction is performed, for example,by the computer 226 (FIGS. 2A-2B) or by the image processing system 406(FIG. 4). Because the first image is a surface image and the secondimage at least partially includes sub-surface image data, subtractingthe first image from the second image produces a sub-surface image. Thethird image thus is a sub-surface image of the subject's skin. In someembodiments, if the second setting completely rejects light having thefirst polarization, the subtraction operation 712 is omitted, since thesecond image already is a sub-surface image.

In some embodiments, the third image is displayed (714) (e.g., in the UI234, FIGS. 2A-2B). In some embodiments, the third image is analyzed andresults of the analysis are displayed (e.g., in the UI 234, FIGS.2A-2B).

In some embodiments, the method 700 further includes a method 720 asillustrated in FIG. 7B in accordance with some embodiments. In themethod 720, a computerized analysis is performed (722) of the thirdimage to determine a skin color of the subject. The skin color is usedto automatically select (724) a recommended cosmetic product.Instructions are transmitted (726) to an automated cosmetic formulationsystem (e.g., the system 500, FIG. 5) to produce the recommendedcosmetic product. The method 720 is performed, for example, by thecomputer 226 (FIGS. 2A-2B) or by the image processing system 406 (FIG.4).

In some embodiments, the method 700 further includes a method 730 asillustrated in FIG. 7C in accordance with some embodiments. In themethod 730, a computerized analysis of the third image is performed(732) to identify a skin feature (e.g., a skin condition) in the thirdimage. An automated comparison of the identified skin feature to acorresponding skin feature in a stored historical image is performed(734). Results of the comparison are displayed (e.g., in the UI 234,FIGS. 2A-2B). The method 730 is performed, for example, by the computer226 (FIGS. 2A-2B) or by the image processing system 406 (FIG. 4).

In some embodiments, the photodetector is calibrated (e.g., using acolor chart positioned in the field of image) and the first and secondimages are corrected in accordance with the calibration.

FIG. 7D is a flow diagram illustrating a method 740 of processing anddisplaying images of skin in accordance with some embodiments. Themethod 740 is implemented at a computer system such as the computer 226(FIGS. 2A-2B) or the image processing system 406 (FIG. 4).

In the method 740, a first image of a subject (e.g., subject 202, FIGS.2A-2C) is received (742). The first image was acquired at an opticalapparatus (e.g., a camera 204, FIGS. 2A-2C) with the subject illuminatedwith light having a first polarization (e.g., light from one or morelight sources 208 as filtered by one or more polarizers 210, FIGS.2A-2C). The first image was acquired with the optical apparatusconfigured to receive light having the first polarization and tootherwise reject light (e.g., an adjustable polarizer 220 was set to a0° rotation with respect to the polarizers 210).

A second image of the subject is received (744). The second image wasacquired at the optical apparatus with the subject illuminated withlight having the first polarization. The second image was acquired withthe optical apparatus configured to at least partially reject lighthaving the first polarization and to at least partially receive lighthaving polarization distinct from the first polarization (e.g., anadjustable polarizer 220 was set to a rotation of greater than 0° withrespect to the polarizers 210).

The first image is subtracted (746) from the second image to generate athird image of the subject. The third image is displayed (748). In someembodiments, the third image is analyzed to identify skin conditions orfeatures and results of the analysis are displayed.

FIG. 8 is a block diagram illustrating a computer 800 in accordance withsome embodiments. In some embodiments the computer 800 is an example ofan implementation of the computer 226 (FIGS. 2A-2B), image processingsystem 406 (FIG. 4), or cosmetic formulation control system 408 (FIG.4). The computer 800 typically includes one or more central processingunits (CPUs) 802, one or more network or other communications interfaces806, memory 804, and one or more communication buses 814 forinterconnecting these components. The communication buses 814 mayinclude circuitry (sometimes called a chipset) that interconnects andcontrols communications between system components. The computer 800 mayalso include user interface hardware 808 comprising a display device 810and a keyboard and/or mouse (or other pointing device) 812. Memory 804includes high-speed random access memory, such as DRAM, SRAM, DDR RAM orother random access solid state memory devices; and may includenon-volatile memory, such as one or more magnetic disk storage devices,optical disk storage devices, flash memory devices, or othernon-volatile solid state storage devices. Memory 804 may optionallyinclude one or more storage devices remotely located from the CPU(s)802. Memory 804, or alternately non-volatile memory device(s) withinmemory 804, comprises a computer readable storage medium. In someembodiments, memory 804 stores instructions for performing all or aportion of the methods 700, 720, 730, and/or 740 (FIGS. 7A-7D). In someembodiments, memory 804 stores the following programs, modules, and datastructures, or a subset thereof:

-   -   an operating system 816 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   a network communication module 818 that is used for connecting        the computer 800 to other computers via the one or more        communication network interfaces 806 and one or more        communication networks, such as the Internet, other wide area        networks, local area networks, metropolitan area networks, and        so on;    -   an imaging control module 820 for controlling an imaging system        (e.g., a system 200, 240, 250, or 300, FIGS. 2A-2C and 3);    -   an image processing module 826 to process acquired skin images        (e.g., images acquired using a system 200, 240, 250, or 300,        FIGS. 2A-2C and 3);    -   an image display module 834 module to display skin images and        data corresponding to skin images;    -   a database of historical images 836 (e.g., for comparison to        newly acquired images); and    -   a cosmetic formulation control module 838 for controlling an        automated cosmetic formulator (e.g., formulator 48, FIGS. 4 and        5).

In some embodiments, the imaging control module 820 includes a polarizercontrol module 822 for automatically controlling an adjustable polarizer(e.g., for controlling the motor 222 via the control board 224, FIG. 2A)and/or an image acquisition module 824 for controlling image acquisition(e.g., using a camera 204, FIGS. 2A-2C).

In some embodiments, the image processing module 826 includes an imagesubtraction module 828 for subtracting respective acquired images (e.g.,in accordance with operations 712 (FIG. 7A), 746 (FIG. 7D) or 768 (FIG.7E)). In some embodiments, the image processing module 826 includes animage calibration module 832.88

Each of the above identified elements in FIG. 8 may be stored in one ormore of the previously mentioned memory devices. Each of theabove-identified modules corresponds to a set of instructions forperforming a function described above. These sets of instructions neednot be implemented as separate software programs, procedures or modules.Various subsets of the above-identified modules may be combined orotherwise re-arranged in various embodiments. In some embodiments,memory 804 may store a subset of the modules and data structuresidentified above. Furthermore, memory 804 may store additional modulesand data structures not described above.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious embodiments with various modifications as are suited to theparticular use contemplated.

1. A method of generating a sub-surface skin image, comprising: at asystem having a memory and one or more processor for processing anddisplaying images of skin: illuminating a subject with polarized lighthaving a first polarization; setting an adjustable polarizer to a firstsetting to admit light having the first polarization onto aphotodetector and otherwise reject light; with the adjustable polarizerin the first setting, using the photodetector to acquire a first imageof the illuminated subject; setting the adjustable polarizer to a secondsetting to at least partially reject light having the first polarizationand to at least partially admit light having polarization distinct fromthe first polarization onto the photodetector; with the adjustablepolarizer in the second setting, using the photodetector to acquire asecond image of the illuminated subject; and subtracting the first imagefrom the second image to generate a third image of the subject.
 2. Themethod of claim 1, further comprising: performing computerized analysisof the third image to determine a skin color of the subject; and usingthe skin color to automatically select a recommended cosmetic product.3. The method of claim 2, further comprising: transmitting instructionsto an automated cosmetic formulation system to produce the recommendedcosmetic product.
 4. The method of claim 1, further comprising:performing computerized analysis of the third image to identify a skinfeature in the third image; and performing an automated comparison ofthe identified skin feature to a corresponding skin feature in a storedhistorical image.
 5. The method of claim 1, further comprising:calibrating the photodetector; and correcting the first and secondimages in accordance with the calibration.
 6. A computer-implementedmethod of processing and displaying images of skin, comprising: at asystem having a memory and one or more processor for processing anddisplaying images of skin: receiving a first image of a subject, thefirst image having been acquired at an optical apparatus with thesubject illuminated with light having a first polarization and theoptical apparatus configured to receive light having the firstpolarization and to otherwise reject light; receiving a second image ofthe subject, the second image having been acquired at the opticalapparatus with the subject illuminated with light having the firstpolarization and the optical apparatus configured to at least partiallyreject light having the first polarization and to at least partiallyreceive light having polarization distinct from the first polarization;subtracting the first image from the second image to generate a thirdimage of the subject; and displaying the third image.
 7. A system forprocessing and displaying images of skin, comprising: memory; a display;one or more processors; and one or more programs stored in the memoryand configured for execution by the one or more processors, the one ormore programs including: instructions to receive a first image of asubject, the first image having been acquired at an optical apparatuswith the subject illuminated with light having a first polarization andthe optical apparatus configured to receive light having the firstpolarization and to otherwise reject light; instructions to receive asecond image of the subject, the second image having been acquired atthe optical apparatus with the subject illuminated with light having thefirst polarization and the optical apparatus configured to at leastpartially reject light having the first polarization and to at leastpartially receive light having polarization distinct from the firstpolarization; instructions to subtract the first image from the secondimage to generate a third image of the subject; and instructions todisplay the third image.
 8. A computer readable storage medium storingone or more programs configured to be executed by a computer system toprocess and display images of skin, the one or more programs comprising:instructions to receive a first image of a subject, the first imagehaving been acquired at an optical apparatus with the subjectilluminated with light having a first polarization and the opticalapparatus configured to receive light having the first polarization andto otherwise reject light; instructions to receive a second image of thesubject, the second image having been acquired at the optical apparatuswith the subject illuminated with light having the first polarizationand the optical apparatus configured to at least partially reject lighthaving the first polarization and to at least partially receive lighthaving polarization distinct from the first polarization; instructionsto subtract the first image from the second image to generate a thirdimage of the subject; and instructions to display the third image.